A floating-point number that tells the gradient descent algorithm how strongly to adjust weights and biases on each iteration . For example, a learning rate of 0.3 would adjust weights and biases three times more powerfully than a learning rate of 0.1.
Learning rate is a key hyperparameter . If you set the learning rate too low, training will take too long. If you set the learning rate too high, gradient descent often has trouble reaching convergence .1
목차 목차 관련 글 ML crash course - Linear regression 관련 절: Gradient descent 역전파 이 글을 인용: “A large learning rate will increase or decrease each weight more than a small learning rate.” Nested Learning: The Illusion of Deep Learning Architectures “Over the last decades, developing more powerful neural architectures and simultaneously designing optimization …” Parameter (machine learning) 이 글을 인용: “In contrast, hyperparameters are the values that you (or a hyperparameter tuning service) supply to the model.” Epoch (machine learning) “The number of epochs is a hyperparameter you set before the model begins training.” Introduction to Machine Learning “Machine learning crash course: If you're ready for an in-depth and hands-on approach to learning more about ML.” Clipping (machine learning) “Gradient clipping forces gradient values within a designated range during training.” Machine Learning Systems “Machine Learning Systems provides a systematic framework for understanding and engineering machine learning (ML…” Recall (machine learning) “True positive rate) is the y-axis in an ROC curve.” Self-training (machine learning) “A variant of self-supervised learning that is particularly useful when all of the following conditions are true:” 배치 크기 “Mini-batch stochastic gradient descent, in which the batch size is usually between 10 and 1000.” 지도 학습 “Supervised machine learning is analogous to learning a subject by studying a set of questions and their corresp…” Generalization (machine learning) “To encourage generalization, regularization helps a model train less exactly to the peculiarities of the data i…” 모델 (기계학습) “Unsupervised machine learning also generates models, typically a function that can map an input example to the …” 기계학습 “2024-08-24 - What's Really Going On in Machine Learning?” Regularization rate “Conversely, reducing or omitting the regularization rate increases overfitting.” Vanishing gradient problem “Increasingly lower gradients result in increasingly smaller changes to the weights on neurons in a DNN, leading…” Training loss “A metric representing a model's loss during a particular training iteration.” Semi-supervised learning “Self-training is one technique for semi-supervised learning.” Example (machine learning) “Labeled examples are used during training.”